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PREFACE 



The primary purpose of this monograph is to facilitate an initial literature review relevant to our 
study granted by the National Science Foundation- “How Reform Works: An Evaluative Study of 
NSF’s Urban Systemic Initiatives.” The three-year evaluative study explores the impact of USI 
programs on student achievement and the learning infrastructure in urban school districts, and will 
develop an inferential causal model linking the Systemic Initiatives drivers and other key elements. 
The study design includes collection and analysis of all 21 USI sites’ quantitative and qualitative 
data, site visits in four districts in each cohort, and the Survey of Enacted Curriculum. This 
monograph provides a basis for the rubric design of the evaluative study- See Appendix A: USI 
Evaluative Study Abstract. 
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ABSTRACT 



What Matters in Urban School Reform 

This review of selected literature examines the National Science Foundation’s standards-based 
systemic reform theory in the context of research reported by science, mathematics, and urban 
education policy experts. This review is specifically concerned with viewing evidence of 
relationships among identified change variables that are associated with fostering high achievement 
in mathematics and science of urban and underrepresented minority students. The authors indicate 
that four categories of variables are useful in studying student achievement: demographic 
information, test data, teacher development, and mathematics and science curriculum. The review 
revealed little researcher agreement regarding variable relationships that might predict a chain of 
influence from policy to classroom practice and finally to student performance. 



INTRODUCTION AND STUDY BACKGROUND 



Response to Access Challenges: 

A History of the National Science Foundation f s 
Systemic Reform Efforts in Urban Schools 

In the 1980's the National Science Foundation (NSF) joined other government and 
policy agencies in the nation's most recent efforts to improve K-12 education by 
improving learning outcomes. Based on evidence of lagging performance by 
American school children on international measures, the Congress joined the 
nation's governors and various discipline-anchored education associations to endorse 
creation of content standards and their implementation through broad-based 
systemic reform. 

The NSF commitment to support improvement in K-12 learning outcomes of 
American students was evident in the creation of the Directorate for Education and 
Human Resources (EHR). In 1990, EHR began formulating strategies to address the 
lack of synchronization between the preparation of K-16 students and projected 
workforce demands. Given anticipated demographic shifts, of particular concern to 
NSF leaders was the underrepresentation of blacks, Hispanics and other minorities 
in the science and engineering workforce (Massey, 1992). By 1992, NSF faced a 
dismal reality in reviewing its 20+ year record of efforts to increase access for 
under-represented populations to careers in the sciences. 



With EHR leadership, NSF revised the 
mission of its Career Access program. The 
Foundation changed its emphasis from 
university level programs to an emphasis on 
pre-collegiate approaches. These approaches 
are designed to "strengthen student science, 
mathematics, and technology literacy through 
curriculum reform, teacher enhancement, 
student enrichment, and other activities" (NSF, 
1992, p. 7). 



The revised vision for NSF's K-12 programs reflected guidance from professional 
groups such as the National Council of Teachers of Mathematics (NCTM), the 
National Research Council (NRC), the American Association for the Advancement 



Students of all races filter out of science 
and engineering so that only a fraction of 
interested high schoolers— L4%— earn 
Ph.D.s. But the minority pipeline , smaller 
to begin with constricts even more sharply 
than that of the total population; only 0.4% 
of minority students emerge with Ph.D.s in 
science or engineering (Culotta & 

Gibbons, 1992, p. 1212). 
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of Science (AAAS), and the National Science Teachers Association (NSTA). These 
groups developed content and evaluation standards, to guide curriculum reform and 
teacher retraining; thereby, framing "what" students should know and be able to do, 
in order to achieve target competence. 

In 1 994, concerns about implementing standards converged with the formulation of 
national education goals by the Governors Association. The governors adopted 
Goals 2000 and supported Congressional action to adopt the Educate America act 
which set the terms for "how" schools would respond to new demands for better 
quality teaching and improved learning outcomes. Congressional demand for 
rigorous, clear content and pedagogical standards signaled significant change in the 
terms of relationships between federal funding agencies and state and local school 
districts. NSF leveraged movement toward systemic reform through grants to 
states and subsequently to urban school districts. 



State Systemic Efforts Set the Stage for Urban Systemic Efforts 

Responsibility for education in U.S. schools resides at the state level. State-level 
policies have increasing influence in three areas: development of content standards, 
state-wide assessment of student learning, and teacher preparation and licensure 
(Blank and Langesen, 1999). States have assumed leadership roles in aligning state 
policy to achieve standards-based reform. In several states, state-level policy guides 
selection of textbooks and other classroom curriculum materials (Blank and 
Langesen, 1999). The Council of Chief State School Officers (CCSSO) serves as a 
clearinghouse to facilitate inter-state communication about progress in development 
and implementation of reform policies. 

With state guidance, local districts are challenged to develop new policies designed 
to up-grade K-12 education, with particular emphasis on increasing instructional 
time spent on improving the quality of mathematics and science learning in 
American schools. In order to comply with higher state standards, school districts 
work to modify policies, which govern graduation requirements and teacher 
qualifications. Emerging student and teacher state assessment programs are aligned 
with new curriculum standards. 

Premised on assumptions that clear, rigorous standards are necessary to improve 
student performance, particularly in mathematics-based subjects, standards-based 
reform defines common expectations for high levels of academic competence. 
Urban, suburban and rural schools are subject to the same standards. School 
districts are challenged to respond to “state established policies [that] define “district 
and school accountability for educational progress of students” (Blank and 
Langesen, 1 999, p. 2). 
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Systemic Reform as an Equity Tool 

Urban systemic reform embraced strategies conceptualized to increase educational 
opportunity for urban poor and minority children. National interest was prompted 
by concerns about a retreat in equity progress, particularly in light of changing 
demographics, as a factor in national economic competitiveness. 

Following a seventeen-year record (1971-88) of achievement gains for African 
American students, 1990 National Assessment of Educational Progress (NAEP) 
scores indicated a reversal of progress, with an increasing gap between the 
achievement scores of whites and African American children. According to 
researchers Smith and O’Day (1991), during the period of progress, emphasis on 
basic skills instruction and testing constituted a national standard for educating 
urban poor and minority students. Minimum competency standards and improved 
social conditions narrowed achievement gaps, but failed to provide access to 
adequate curriculum for competitive achievement. The problem of the continuing 
gap is exacerbated by social realities that followed the period of civil rights progress. 
During the final decades of the 20 th century, hyper segregation and high poverty 
rates characterized conditions in most urban communities (Orfield and Ashkinaze, 
1991; Denton and Massey, 1993). Urban social realities combined with the basic 
skills curriculum offered in urban schools limit access to complex knowledge and 
skills assessed by the emerging standards-based state assessments (Smith and 
O’Day, 1991). 

NSF and other federal agencies directed funding to energize comprehensive change 
at all levels of the education enterprise, including the school district level, the school 
building level, and in the relationships between schools, associated universities, 
industry, and other societal partners. To qualify for funding, local education 
agencies adopted systemic reform terms consistent with those outlined by policy 
researchers (Smith and O’Day, 1991; Fuhrman, Elmore and Massed, 1993). These 
terms are expressed as reform “drivers.” According to the NSF, there are “six 
critical developments that drive systemic change” (Westat*McKenzie, January 
1998, p. 13). 

School districts prepared to redesign their operations to implement systemic change 
based on NSF’s Driver model. NSF engaged districts through contractual 
relationships that established terms for sustained activities, based on "systemic and 
comprehensive approaches.” The Urban Systemic Initiative (USI) focused resources 
on school districts in communities with high numbers of children living in poverty, 
and the lowest levels of academic achievement. The USI "was established [to 
support] sustained science and mathematics education reform in the nation's largest— 
and poorest— urban centers" (Westat*McKenzie, October 1998, p. 7). 
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Six Drivers for 
Educational System Reform 

1. Implementation of comprehensive, standards- 
based curricula as represented in instructional 
practice, including student assessment, in every 
classroom, laboratory, and other learning 
experience provided through the system and its 
partners . 

2. Development of a coherent, consistent set of 
policies that supports: provision of high quality 
mathematics and science education for each 
student; excellent preparation, continuing 
education, and support for each mathematics and 
science teacher (including all elementary 
teachers); and administrative support for all 
persons who work to dramatically improve 
achievement among all students served by the 
system. 

3. Convergence of the usage of all resources that are 
designed for or that reasonably could be used to 
support science and mathematics education— 
fiscal, intellectual, material, curricular, and extra- 
curricular— into a focused and unitary program to 
constantly upgrade, renew, and improve the 
educational program in mathematics and science 
for all students. 

4. Broad-based support from parents, policymakers, 
institutions of higher education, business and 
industry, foundations, and other segments of the 
community for the goals and collective value of the 
program, based on rich presentations of the ideas 
behind the program, the evidence gathered about 
its successes and its failures, and critical 
discussions of its efforts. 

5. Accumulation of a broad and deep array of 
evidence that the program is enhancing student 
achievement, through a set of indices that might 
include achievement test scores, higher level 
courses passed, College admission rates, college 
majors, Advanced Placement Tests taken, portfolio 
assessment, and ratings from summer employers, 
and that demonstrate that students are generally 
achieving at a significantly higher level in science 
and mathematics. 

6. Improvement in the achievement of all students, 
including those historically underserved. 



Following a 1996 retreat convened by 
National Science Foundation program 
officers, the Foundation issued detailed 
guidelines for building equity into NSF 
systemic reform initiatives. The guidelines 
clearly outline expectations that “equity is an 
essential and inherent component of high- 
quality mathematics and science at all levels” 
(Westat*McKenzie, 1997, p.6). The 
guidelines call for accommodating wide 
variation in approach to ensure that 
subgroups of people do not experience 
systemic discrimination in process, 
opportunity nor outcomes. This guidance 
interprets NCTM and NRC standards as 
implicating “a need to go beyond equal 
access and treatment to understand various 
learning styles, and motivations among 
students.” It also calls for making 
“adaptations to meet the needs of special 
students, and to provide opportunities for 
those students interested in and capable of 
moving beyond the basic program” 
(Westat*McKenzie, 1997, p.18). The 
Foundation established expectations that 
outcomes of systemic reform would include 
proportional representation among high 
achievers of students from the range of 
social, ethnic, racial, and economic groups 
comprising the American population. 

Evaluating USI implementation, then, 
requires viewing local district outcomes 
within the context of each district’s 
adaptations of its state’s standards-based 
curriculum and assessment systems. 
Determining the alignment between the 
urban district’s curricula and that of the state 
must incorporate a view of the district’s 
adaptive features that support success for 
diverse learners. 
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Scope of the Current Literature Review and Concerns of the Related Study 

NSF's approach is considered logical, clear and comprehensive by many researchers. 
Others studying reform implementation find wide, perhaps unwieldy, variation in 
state and local reform initiatives (Knapp, 1997; St. John & Pratt, 1997; Long, 1996). 
The literature is reviewed to relate NSF's implicit standards-based reform theory to 
ideas and issues advanced by science, mathematics, and urban education scholars. 

While the NSF-identified reform drivers are key research variables, the literature 
review indicates that variation among the perspectives of key stake-holders, such as 
teaching and learning experts, about desired teacher and student behavior within a 
standards-informed learning environment implicates equity concerns as an 
additional and influential variable. The NSF Equity Implementation guidance was 
issued in January 1998, and 22 of the 28 USTs were funded prior to 1996. 

As this study investigates the interaction of "standards" (what all students should 
know and be able to do) and process variables (curriculum content and instructional 
process), as evidenced by benchmarks (grade level achievement "snapshots"), the 
emerging variable interaction model should be sensitive to critical variation in policy 
interpretation and implementation. Data such as the content and process claimed as 
professional development, as well as the beliefs and backgrounds of system 
administrators and teacher trainers yield important insight about relationships among 
change variables. 

Other NSF commissioned studies also inform this research. Conclusions from the 
initial analysis of the relationship between state standards and assessments indicate 
that researchers believe their emerging study strategies are useful in viewing 
synchronization between content planned and content assessed. Viewed with 
results of the Survey of Enacted Curriculum, alignment studies that incorporate 
evidence of urban standards-based curricula adaptations charted to support success 
of urban students will likely provide a useful indicator of the implementation status 
of content standards in urban districts. 

This literature review necessarily reflects the range of responses reported by 
researchers to the underlying mission and approach to systemic reform. This study 
of How Reform Works is concerned with the relationships among change variables, 
particularly as those relationships are associated with positive changes in student 
achievement and system outcomes. In this regard, the current review attempts to 
examine the NSF’s theory of reform, against the background of research reports that 
reflect theoretical and application-driven insights about fostering high achievement 
of urban and underrepresented students, particularly in mathematics and science. 
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NSF’S REFORM THEORY AND RESEARCH VARIABLES 



What Variables are Used in Research on Mathematics and Science Achievement 
of Urban and Underrepresented Students? 

Based on a survey of current literature, variables found to influence performance in 
science and mathematics fall into four categories: demographic information, student 
performance information, mathematics and science curriculum, and teacher 
professional development. 

By the year 2000, Anderson (1990) indicates that approximately one-third of the 
nation will be minority. It is expected that almost one-third of the nation's jobs will 
require a college education and that nonprofessional jobs will require increasing 
skills in mathematics and reasoning. Anderson views mathematics as an economic, 
social, and psychological enabling force and a filter that allows students the option 
of an unlimited number of careers both professional and nonprofessional. Two case 
studies discussed in the journal article underscore the importance of educators' high 
expectations and encouragement of minority students in mathematics. 

Rech and Stevens (1996) in an article entitled “Variables related to mathematics 
achievement among black students” appearing in The Journal of Educational 
Research reported that extensive research has been compiled to compare the 
mathematical achievement of black and white students. The findings of these 
studies have consistently shown that black students have been significantly 
outperformed by non-black students at all grade levels. Blacks, as a group, have not 
been expected to perform well in mathematics leading to learned helplessness and 
lower teacher expectations. The authors, Rech and Stevens, designed a study to 
determine the effect of four specific characteristics on the mathematical achievement 
of fourth- and eighth-grade black students. The goal of their research was to obtain 
information to use in developing instructional interventions to improve achievement 
for this group of students. Strict scientific methods were used in gathering and 
analyzing the data. 

The results showed that there were significant differences between the fourth- and 
eighth-grade students. Attitude and economic status were found to be predictors of 
mathematical achievement in fourth graders. The authors suggest that educators 
need to take steps to improve negative attitudes. Students need to be made aware of 
the relevance of science and math to their everyday lives and its value in society. It 
is also important that students are given many opportunities to be successful in 
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science/mathematics to counteract these negative attitudes. Given that economically 
disadvantaged black students are more likely to fail, special social programs are 
needed to address this situation. 

It was interesting to note that even though the eighth graders in the sample possessed 
more negative attitudes toward math than the fourth graders, attitude was not a 
significant predictor of achievement. The research study found that learning styles 
and gender were predictive factors for black eighth-grade achievement. Educators 
need to learn to teach to a variety of learning styles with an emphasis on the active 
engagement of students through a variety of concrete, hands-on, learning 
experiences. Mathematical learning activities should be relevant and provide a 
global perspective. The study also revealed that eighth-grade black males achieve at 
a lower level than the females. Two recommendations to rectify this situation are to 
bring black male role models into the classroom and to design instructional 
strategies that specifically target black males. 

The students in this sample had a mean percentile score of 47.2 in the fourth grade 
which dropped to 37.0 in the eighth grade compared to scores of 60.6 and 57.7 for 
white students, respectively. Rech and Stevens indicate this is unacceptable. In an 
advanced nation such as the United States, one cultural group’s achievement should 
not be allowed to lag so far behind. They argue that educators have the ability to 
positively impact at least three of the four predictors of achievement. Teachers can 
positively impact the attitude of fourth graders by making mathematics relevant to 
the students’ everyday lives and fun. They can design activities that promote 
positive learning experiences and allow all students to be successful. Through a 
partnership with school and the community, positive black male and female role 
models should be brought into the classroom and mentoring programs established. 
Teachers need to be educated in different learning styles and be given the support to 
provide active, hands-on activities that will engage all students. The only factor that 
educators cannot correct on their own is the effect of poverty on children’s learning. 
A concerted effort involving the community and social service agencies in 
partnership with the schools is necessary to positively impact economic status. 

Gross (1993) reported a study in the Journal of Negro Education with data derived 
from work done in 1988. He noted that the school district used for data collection 
had placed a priority on the education of minority students and the equitable 
representation of these students in advanced level classes. Also, minority students in 
this district had outscored minority students nationwide and in many cases had 
scored better than all students in their age/grade level in the test publisher’s norming 
group. Even so, there were large differences in the scores of minority and majority 
students in this school district. 
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Following are the major findings from Gross's study: 

❖ This school district assesses behavioral objectives at each grade level. In 
first grade, the students start out virtually the same with almost all students 

on grade level. Significant differences appear by 
the end of second grade and the gaps widen every 
year. At the end of sixth grade, 40% of whites 
were above grade level compared to just over 10% 
of blacks, and the percentages of students working 
below grade level were about 20% and 52% for 
whites and blacks, respectively. 

♦> African American students scored on average 
15 NCE points below their white counterparts on 
the California Achievement Test (CAT) in fourth, 
sixth, and eighth grades and 20 and 4 NCE points 
below the Asians and Hispanics in their school 
district, respectively. 

❖ The author performed factor and regression 
analyses to investigate the relationship of class 
performance, attitudes, and behaviors of students 
to performance on the CAT. The variable that was 
found to account for approximately one-half of the 
variance in scores was students' performance in 
the mathematics curriculum. 

♦♦♦ Interviews of school staff revealed that societal factors were blamed for the 
discrepancy in the achievement between minority and majority students. 
Teachers were asked to rate students on several aspects of classroom behavior. 
Findings were that black students were judged less studious by their teachers as 
well as less prepared and careful in their schoolwork. In the fourth- and sixth- 
grade samples of teachers, majority students were rated significantly higher in 
overall ability when compared to their minority counterparts. 

Gross notes in her study that strong relationships were observed between students' 
performance in the school's mathematics curriculum and their mathematics 
achievement. She concludes that an early emphasis on all aspects of the 
mathematics curriculum will lead to higher achievement for minority students. She 
also feels that school administrators and teachers perceive that minority students are 
less interested in and less capable of success in mathematics. She recommends that 
a key component of improving mathematics achievement for minority students lies 
in training teachers to provide mathematics instruction using a constructivist 
approach although no evidence is presented to support this assertion. Gross 
emphasizes that teacher in-service must educate teachers using the same methods 
that they will be expected to employ in their classrooms. Emphasis should be placed 
on teachers' questioning and listening abilities. Gross speculates that attitudes and 
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beliefs of teachers about who is capable of doing math will change as interaction and 
collaboration between students and teachers increase. 

The author seems to be accounting for the underachievement of minority students as 
partly due to teachers’ stereotyping and preconceptions. The author states that 
constructivism takes the pressure off of the teacher to ’’cover the curriculum”. One 
would hope that teaching using a constructivist approach would allow all students to 
master concepts and skills so that they can be built upon rather than retaught every 
year. 

Davidson and Kramer (1997) make the case that major change is necessary in three 
areas to empower all students to achieve in mathematics. The easiest of these areas 
to change is curriculum. The authors give several examples demonstrating how the 
mathematics of a culture can be used to explore a mathematical concept that is 
developmentally appropriate, and conversely, how the study of a culture can present 
opportunities to use mathematics within that context. 

Secondly, the authors call for teachers to take a hard look at how we teach and our 
attitudes and perceptions of our students. They emphasize the importance of 
students’ construction of knowledge and the use of cooperative learning. Teachers 
must actively encourage positive inter-group interactions, which will help to reduce 
prejudice and level the multicultural playing field. 

Davidson and Kramer make an interesting point that students’ preparation for school 
depends on how well their culture matches the culture of the school they attend. 
Children who come from different cultures need to be taught the rules. Furthermore, 
to have more opportunities to succeed in the United Stages, students must learn the 
rules of the middle- to upper-class white culture that is in power. This theory has 
definite implications for urban inner city schools and our society as a whole. 

There is a discussion of whether the emphasis should be placed on traditional 
technical skills or the currently popular conceptual or developmental approaches. 
Davidson and Kramer present evidence that no one approach will serve all children. 
Necessary components of successful teaching are validation of students, demand for 
excellence, collaborative learning, and the learning of skills within the contexts of 
problem solving and critical thinking. 

Thirdly, the culture of our classrooms needs to change in order to truly value all 
cultures. This must start with the teacher and emanate from him/her. By actively 
including in our lessons a variety of cultures, we show the students that differences 
are valued. Rather than making our classrooms into a 'melting pot' where all 
students are treated the same, we should investigate and celebrate our differences. 
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The premise of the authors appears to be valid; but putting these major changes into 
effect concurrently in science/math classrooms seems to be a daunting task. Take, 
for example, the call for cooperative learning. This is a practice that most teachers 
understand, but as Davidson and Kramer report "...the typical American classroom, 
at any level, is oriented toward competitive individualism" (1997, p. 137). 
Competitive individualism, right or wrong, is a basic tenet of middle- to upper-class 
white culture. Many parents and students are opposed to the use of cooperative 
learning and our current emphasis on grades is definitely not in keeping with the 
objective of a classroom working together toward a common goal. Also, teachers 
need to be given the tools to teach about a variety of cultures. This is not a simple 
matter as educators teach best when they know their subject matter and feel 
competent, and without appropriate training, teachers may be out of their comfort 
zone. This is not to suggest that the authors' suggestions are not valid or even 
feasible, only that the changes called for are dramatic and not easily attained. 

Both the Davidson and Kramer (1997) and the Gross (1993) articles address the 
importance of professional development and variables to be considered in this area. 

Archibald (1995) undertook a longitudinal study to compile data related to school 
choice and magnet school outcomes. The longitudinal design comparing second and 
fifth grade scores was conducted to try and minimize the confounding influences of 
self-selection bias and focus on achievement. Sample sizes of 3,890 (Cohort 1) and 
4,680 (Cohort 2) Milwaukee Public School (MPS) students were used to expand the 
sample size and increase generalizability of the results. The dependent variable was 
the fifth grade mathematics and reading scores for the Iowa Test of Basic Skills. 

The main independent variable was the magnet variable. Other independent 
variables were transfer status, free lunch status (used as a socioeconomic status 
control variable), second grade achievement, and race. Multivariate linear 
regression analysis was employed on the students' Normal Curve Equivalent (NCE) 
scores. 

The study found that students in the magnet schools scored significantly higher in 
both reading and mathematics than both the transfer and non-transfer students. The 
study also shows that the non-transfer students' scores were higher than the transfer 
students' scores, although when these groups were broken down by race, there was 
very little difference between the scores. When the author analyzed the results 
controlling for prior achievement and free lunch status, the magnet school students 
showed an improvement of an average of three NCE points while the transfer 
students had lower scores of one to two NCEs. It was interesting to note that 
students who are not eligible for free lunch will on average outscore their full free 
lunch counterparts by five to six NCE points, equated on prior achievement. 
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Archibald's study clearly shows that the scores of magnet school students are higher, 
but further analysis was deemed necessary to determine why. In a previous study, 
the author had found that there was not a significant difference between the class 
size in magnet and non-magnet schools (27 versus 28 students on average) and that 
there are not significant resource differences between magnet and non-magnet 
schools. Other differences suggested are level of parental involvement, school 
climate, quality of teachers, and student self-selection. The author calls for further 
research to determine the factors that may account for differential performance. 
Archibald states that "this study adds evidence to the proposition that school choice 
policies create conditions which pay off in higher student achievement" (p. 167). 

There appears to be a contradiction inherent in the author's last statement. Assuming 
that transfer students choose to go to school outside of their neighborhood, why are 
their test scores lower that their non-transfer counterparts? Even though the 
difference is not large, the scores of black non-transfer students are consistently 
higher than the black transfer students. The author states that the majority of black 
transfer students are from the racially isolated inner city region of Milwaukee and 
that there are about fifteen schools in the inner city that have few or no non-black 
children. It would be interesting to see statistics comparing student scores from 
these schools with scores of transfer students from the same area and see if any 
patterns emerge. 

The author's purpose for this study was to examine the effects of school choice, and, 
more specifically, magnet schools’ impact on student achievement. His data 
revealed some very disturbing statistics which he did not address. Table 3 indicated 
the mean fifth-grade NCE scores for magnet, transfer, and non-transfer students 
broken down by race. In all categories, white students outscored their black 
counterparts by thirteen to nineteen NCE points with the largest differences in the 
magnet school students. Not one subgroup of black students scored above the mean 
of fifty, whereas all subgroups of white students scored above. Also, the scores of 
black magnet school students did not have as large an increase over the scores of 
their black non-magnet school counterparts when compared to the scores of the 
white students. The larger question here seems to be: Why are the scores of black 
students in the same school settings significantly lower than the scores of their white 
counterparts, and what can be done to improve their performance? 

The ethnographic study performed by Walker and McCoy (1997) of seventeen 
African American students selected from a high school in a small city with an 
enrollment of 1250 is necessarily limited. The authors specify that thirty percent of 
the school's population is African American but nothing is told about the 
socioeconomic makeup of the school or the students interviewed. Some important 
points did surface from the interviews, however more research is necessary to 
determine the generalizability of the findings. 
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It was obvious from the interviews of these students that the teacher did play an 
important role in how the students felt about math class. The students' perceptions 
of whether or not their teacher cares about them seems very important to them. 
Teacher characteristics cited which indicated teacher interest included sitting 
African American students in the front of the classroom, encouraging them to 
participate and not be silent, and relating to the students as individuals. The students 
who feel they have a personal relationship with their teacher are more confident and 
seem to work harder in the classroom. At this school there is a black math teacher 
of whom the students were particularly fond. Although the students did not berate 
white teachers, their comments indicated that they felt many white teachers are 
unable to relate to them culturally, and some students felt that white teachers are 
more interested in good behavior than in the African American students as 
individuals. If they behaved well they were virtually left alone, whereas if they 
misbehaved, they received attention from the teacher, albeit negative attention. 

The authors also found a correlation between parental involvement and mathematics 
achievement. Students who were performing well in their math classes cited their 
parents as having a positive influence on their school achievement during the 
interviews. Three students who were failing math either didn't mention any family 
members as positive influences or berated them for their lack of encouragement. 

This study helps to substantiate the findings of other research that has been done in 
this area. The authors' interviews indicate that: 

• The classroom must be a positive setting where there is genuine caring and 
students are expected to learn. 

• Family support is an integral factor in student achievement. 

• African American students may be stereotyped by their athletic abilities 
with school achievement taking a second place. 

• African American students may fear standardized tests and develop an 
apathetic attitude as a defense mechanism. This supports the notion of 
learned helplessness. 

• Many African American students may not realize the potential avenues that 
a solid mathematics background opens up for their future. 

It is good to explore the opinions of African American students in our search for 
answers. Determining how students feel about science/mathematics and their school 
experiences related to these areas may give educators insight into how to include 
African American students in the academic community. It would be helpful to see 
this ethnographic study expanded to students in inner city urban schools and also 
high school students in the pre-algebra level. 
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Tobias (1992) points out that his recommendations for improving mathematical and 
science achievement are applicable to all students but are especially crucial for 
African American students whose achievement is lagging far behind. Tobias feels 
that teacher education programs in mathematics and science have not changed 
significantly, but that strides are being made on the level of in-service teacher 
training. It is important that teachers are taught in the methods in which they will be 
expected to teach. They need extensive experience with problem solving in math 
and science. 

Another important aspect of improving mathematics and science achievement is 
making these subjects relevant to students' everyday lives. Even though it is obvious 
to the teacher that mathematics and science are pervasive in our lives, students must 
be shown and taught this concept. The use of projects, activities, and games to 
highlight this makes math more interesting and more meaningful. Arguably the 
most important change to be made is in raising the self-esteem of the African 
American student in the area of mathematics and science. This particular goal can 
be accomplished by providing students with as many opportunities as possible to 
succeed in computation, problem solving, and comprehension within a structured 
environment. Mathematics and science should be taught in progressive steps with 
constant monitoring of students' mastery. African American students must also be 
taught about the significant original contributions made by mathematicians and 
scientists of their own race and should be provided with classroom role models. 

The suggestions that Tobias makes are relevant and important. One of the most 
frequently asked questions in the middle school math classroom is "Why do I have 
to learn this and when will I use this knowledge?" Rather than answering this 
question, teachers need to show students the relevance to their everyday lives of the 
content being studied. Projects and activities that are meaningful to the students 
should be used to teach the curriculum rather than teaching topics in isolation. 

Tobias indicates that much has been written about the effect of teacher expectations 
on student achievement, but equally important is students’ expectation of their own 
capability. It is imperative that teachers work to raise students' mathematical self- 
esteem, as the accepted popular convention is that certain individuals are 'bom' to 
be good in math and science and the rest of us are not genetically predisposed. 
Students of all races and genders need to be provided with role models disproving 
this myth. 

Tobias’ recommendations could help improve the mathematics and science 
achievement of all students. However, to assure the success of African American 
students, we must concentrate on getting highly qualified math and science teachers 
into our urban and economically disadvantaged schools. 
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Robinson (1996) supports the Tobias position. Although writing in the journal 
Urban Education , Robinson’s premise is that the future of our nation and its ability 
to compete in a technological society are contingent on giving all students the 
opportunity and ability to be competent in mathematics and science. The author 
specifically targets the need for reform in our nation’s urban schools. Robinson cites 
data compiled from the 1992 National Assessment of Educational Progress (NAEP) 
exams which indicated that students in disadvantaged urban areas consistently 
achieved significantly lower proficiency levels than the national averages at all 
grade levels tested. Robinson suggests that one of the causes of their subpar 
performance is the effects of being bom into poverty. A secondary cause indicated 
is the lower expectations for these students held by the education community. 

Robinson expresses three guiding principles that must be met for effective reform in 
science/mathematics achievement in the urban setting. First, the social and 
economic poverty in urban areas must be addressed. Schools alone cannot take on 
this task; the coordination of social agencies and the entire community is imperative. 
Secondly, all students should be expected to learn and to achieve at high levels. The 
mathematics and science curriculum for urban students should be challenging and 
exhibit high standards. If the standards and expectations are raised for these 
students, achievement will be raised. Thirdly, assessment reform is necessary. 
Robinson states that the purpose of assessment should be to enhance student 
achievement. Assessments should evaluate the effectiveness of the curriculum, 
highlight roadblocks to learning, and encourage higher-order thinking skills. When 
used correctly, assessment should also elevate student confidence. 

The need for research specifically related to the mathematics achievement of African 
American students in the United States is highlighted by disturbing and increasingly 
dismal statistics. Over one-third of African American students drop out of school 
before finishing high school. There are more African American males in jail than in 
college, and African American males outnumber whites in jail. In the 1990's, civil 
rights leaders argued that mathematical literacy is necessary for the advancement of 
African Americans, and minority students cannot continue to be tracked out of 
gatekeeper classes such as algebra and calculus. Giving underrepresented students 
access to higher level math increases their chance for educational and economic 
success. (Ladson-Billings, 1997) 

Ladson-Billings’ research (1997) has focused on successful teachers of African 
American students. She believes that in order to change the achievement levels of 
African American students, the pedagogy must be changed. Ladson-Billings 
encountered a "pedagogy of poverty" in her research - students being taught math 
through routine teaching acts that failed to engage or interest them. She found that 
the teachers who taught in this manner tended to possess one or more of the 
following characteristics: 
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• they themselves did not perform well in school, 

• they approach teaching systematically or as a set of rules, 

• their main goal is to manage or control their students, 

• they hold low expectations for their students, and 

• they are unlikely to pursue intellectual pursuits themselves and are unwilling 
or unable to change their teaching techniques. 

In contrast, exemplary teachers of mathematics develop a pedagogy designed to 
ensure high mathematics achievement among African American students. Following 
are principles necessary to this pedagogy: 

• Students are expected to display high levels of competence and to succeed. 

• Teachers use students' prior knowledge to build on to learn new knowledge. 

• Teachers and students are engaged in mathematics learning during the entire 
class period. 

• All students are challenged and expected to perform at levels traditionally 
considered to be beyond their capability. 

• Teachers know their students well and form strong relationships with them. 

The teaching practices necessary for high achievement in mathematics advocated by 
Ladson-Billings should apply to all students. The Third International Mathematics 
and Science Study (TIMSS) (U.S. Office of Education, 1996) revealed that U.S. 
school children continue to lag behind students in other highly technological nations 
in mathematics and science achievement. The reasons for these lags are multiple - 
teachers without adequate preparation in mathematics and science, unimaginative 
approaches to teaching, teacher misassignment, poorly constructed textbooks (p. 
698). These inhibitors to mathematics achievement can be found in most school 
districts to differing extents but are more likely to be pervasive in urban, low-income 
school districts. Ladson-Billings makes a good argument for what needs to be done 
and what our math classrooms should look like, but does not give any specific 
strategies for how we should get there. Instead, she calls for further research and 
documentation of successful practices of teaching mathematics to African American 
students. 

Factors associated with under-representation of African Americans in mathematics 
and science is reported in Journal of Negro Education by Powell (1990). Many 
African Americans perceive themselves as incompetent in the areas of mathematics 
and science. Society's views of scientists and mathematicians as highly intellectual, 
strange, and socially inept make it easy for students to justify not wanting to go into 
such careers. African American college students are over represented in careers 
such as education, social sciences, and humanities which traditionally are lower- 
income occupations. Powell questions whether this is due to cultural differences in 
what is valued or perceived incompetence in fields calling for mathematics and 
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science. The author hypothesizes the latter, although she acknowledges the lack of 
scientific research in this area. 

According to Powell a widely held view “that the mean IQ of African Americans 
falls in the low average range” has led to lower expectations for this cultural group 
in subject areas perceived to be highly intellectual by whites and African Americans 
alike. Learned helplessness is a self-fulfilling prophecy. Scientific studies have 
shown that many African American students receive poor preparation in 
mathematics and science early in their careers and therefore experience repeated 
failure and do not master the basic concepts of these subjects. "Upon encountering 
continuous failure individuals may stop trying, not only in the setting where their 
failure was initially elicited but also in other settings where there might be a better 
chance for success" (1990, p. 294). 

The author also discusses two other factors associated with learned helplessness. 

The first is overcrowded conditions in the home. Powell states that "When there is 
an absence of privacy due to the presence of too many people in the household, 
individuals tend to develop a sense of lack of control..." (p. 295). She also relates 
this to large class size and lack of individual attention. The other causal factor 
discussed is uncontrollable noise. Since mathematics and science are more complex 
subjects, noise pollution seems to negatively impact learning in these subjects to a 
greater extent. 

Powell recommends several strategies to counteract the detrimental effects of 
persistent failure, crowded living and learning conditions, and noise pollution. First 
and foremost, African American students must be given the opportunity to succeed 
on difficult math and science tasks. Clinical studies have shown that the pairing of 
aversive and pleasant stimuli will help improve the symptoms of helplessness over a 
period of time. Powell also recommends that mathematics and science classrooms 
should provide environments that are quiet, pleasant, and orderly. It is also 
important that there is optimal student-teacher interaction. Powell feels it is 
essential that high school counselors encourage students to take a full load of 
mathematics and science courses. Finally, the author advocates that funding be 
made available for further research related to African American career choices. In 
regards to achievement variables, Powell’s contribution seems to be associated with 
“learned helplessness” and having high expectations for students. 

Malloy (1997) indicates the results of the 1992 NAEP showed some improvement in 
the computational skills of African American students, but their performance still 
lags far behind that of the general population. Learning mathematics necessitates 
becoming part of the mathematics community and valuing the importance of 
thinking and doing math. Educators need to study the unique characteristics of the 
African American culture in general and African American students in particular to 
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make instructional recommendations for including these students in the mathematics 
community. 

Most African Americans have strong ties to their culture, which has an effect on 
their mathematics learning. Mainstream culture shapes students' perceptions of 
themselves and their abilities. Malloy (1997) points out that just as many African 
Americans have exceptional athletic abilities and positive social reinforcement 
encourages dreams of their being the next Michael Jordan, there has been an equal 
negative reinforcement of the science and mathematics achievement of African 
American students. The media continually shows the achievement gap between 
these students and the general population resulting in African American children 
being tracked into lower-level math classes early in their schooling. Negative 
perceptions lead to lower expectations of mathematics abilities by the students, their 
parents, and society as a whole. (Malloy, 1997) 

Educators must not allow African American students to dissociate themselves from 
the mathematics learning process. By teaching African American students on the 
basis of what is known about their culture and preferences for learning Malloy 
indicates, hopefully, science and mathematics achievement will be improved and 
expectations raised. Malloy (1997) indicates that African American students learn 
best through direct contact with their teachers and peers. Therefore cooperative 
rather than competitive learning is an effective teaching strategy. It is also important 
that interaction, trust, active engagement, and meaningful relationships are all part of 
the math classroom, and teachers should use student life experiences in their 
instruction. Giving African American students' opportunities to express themselves 
through writing and oral presentations is also consistent with their culture and 
learning style. 

A recent study of African American students who were successful in mathematics 
despite being a part of a community where low achievement was the norm revealed 
the following characteristics: 1) positive family, school, and community interaction; 
2) students sought help when they encountered difficulties; 3) high self-esteem and 
self-efficacy and a desire for mathematics achievement; and 4) participation in 
activities that opened up opportunities. Malloy recommends that math and science 
educators integrate culturally responsive educational practices into their classrooms 
based on empirical findings, and then study the effects of these changes on African 
American student achievement. To include African American students in the 
science/mathematics community, we must change how they perceive their abilities 
and teach them in a manner compatible with their learning styles. 

It is interesting but not surprising that studies of skills needed to be effective 
problem solvers undertaken over the last twenty years have focused on majority 
students. Skills identified were basic computational skills, analytical reasoning 
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ability, field independence, and self-esteem. Research on learning preferences of 
African American students show that they prefer holistic reasoning to analytical 
reasoning and tend to be field dependent. African Americans' self-esteem in the area 
of science/mathematics is generally low due to lower expectations of parents, 
schools, and the community as a whole. It is easy to see that the traditional methods 
of teaching using an expository style that is linear and analytical is in direct conflict 
with the learning style of many African American students. Science/mathematics 
educators need to encourage diverse methods of approaching and solving problems. 
Mathematics also needs to have meaning for all students so that they can construct 
scientific/mathematical knowledge based on experiences that are relevant to their 
lives. Malloy indicates that the teaching practices encouraged by the National 
Council of Teachers of Mathematics (NCTM) are necessary to improve mathematics 
achievement of all United States students. These practices help African American 
students also, but specific strategies must be developed to assure that African 
American students feel capable and become an integral force in the mathematics 
community. 

In summary, it seems that student demographics (specifically, gender and ethnicity), 
classroom climate, pedagogical practice, mathematics and science curriculum, 
teacher professional development and aligned curriculum coupled with appropriate 
assessment concerns constitute the primary areas for variable identification. 



Researchers 9 Responses to the Theory of Systemic Change 
NSF 9 s Primary Change Driver: Standards-Based Curricula 

Both content and pedagogical changes are implicated in the theory of standards- 
based curriculum reform. The success potential of USI is related to a level of 
coherence between the logic and intentions of standards-based content and 
pedagogical approaches. A key concern is whether the standards dictate sensible 
behavior at the classroom level. What indicators define successful implementation of 
standards-based curriculum? What measures best reflect individual student 
achievement? What indicators define success in school buildings and in school 
districts? Responses to these questions drive strategic interactions that determine 
systemic reactions which shape how teaching and learning occur. Responses to 
these questions also ultimately express values, which are embodied in ends sought 
by reform (Howe and Moses, 1999). The answers to these questions might also 
signal critical differences in approaches advocated by various stake-holders. 

The mandate to set more rigorous standards originated with “end” concerns about 
promoting economic competitiveness. To gamer intellectual capital, American 
students must rank among the best educated in the world. Students’ scores on 
standardized tests are the traditional measures that indicate whether American 
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schools are successful in covering selected mathematics and science topics with the 
same depth and breadth as the topics are covered in the schools of competitor 
nations. At the state level, standards-aligned assessments monitor students’ abilities 
to demonstrate content knowledge by performing more complex evaluation tasks. 
When test scores rise, schools are labeled successful; when test scores stagnate or 
decline, schools are labeled as failures. Failing students must spend more time on 
tasks, by attending school for longer periods. Failing schools require staff 
development, reorganization, or reconstitution. In this paradigm, the curriculum is 
fixed and performance on standardized tests indicates students’ achievement, as a 
reflection of system accountability (Hershberg, 1999). 

Standards-based education in another of its earliest conceptualizations established 
development of thinking and problem-solving abilities as the primary objective of 
education, providing mechanisms for "top-down support for bottom-up reform" 
(Darling-Hammond, 1994, p.18). Acknowledging application and authentic 
performance as the crucial by-products of the process, many reform advocates, in 
this tradition, assign priority to reorientation of the best example of teaching from 
teacher, as expert, to student, as questioner. Reform advocates, particularly faculty 
working within university schools of education, believe that American children will 
think and solve complex problems more effectively, if schools generally require 
more intellectually demanding work and expect higher performance (Cohen, 1995). 
Many education faculty researchers support approaches established by a line of 
education Progressives who emphasize development of complex cognitive and 
performance abilities that are demonstrated in a variety of ways, including authentic 
demonstrations, in non-test contexts (Dewey, 1916; Bloom, Hastings, & Madaus, 
1971). In this paradigm, students and teachers interact as problem-solvers. 
Knowledge is not fixed, but is discovered through interactions with many types of 
experiences, including solving non-routine and open-ended problems (D’Ambrosio, 
Johnson & Hobbs, 1995). Student work demonstrates achievement measured 
against context embedded performance standards, which students receive as rubrics 
or scoring guides to guide their work (Seidel, 1991; Schwarz, 1994). 

Following the lead of NCTM and NRC, the NSF theory of reform juxtaposed 
content-based standards and assessments with inquiry learning. There can exist 
tension in this “marriage.” Central to the shift from teacher-expert approaches to 
student- and teacher-problem-solver approaches lie key cultural issues, such as what 
body of information constitutes defining knowledge. In the Progressive tradition, 
many educators believe that schools should facilitate self-affirming connections 
between students and curriculum (schooling experiences) as opposed to relating 
learning to knowledge and schooling customs established in earlier periods to reflect 
dominant cultural beliefs and purposes (Giroux, 1997; Apple, 1983; Friere, 1973; 
Dewey, 1916). 
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In contrast, Western traditionalists believe that education should shape the ideas and 
ideals of American citizens to reflect dominant cultural values (Sowell, 1992; 

Hirsch, 1987; Bloom, 1987). Content-based reform asserts 
very specific curricular topics, as well as types and levels of 
information required. Anticipating debate about the nature 
of curricular content “within the context of content-driven 
systemic reform,” systemic reform strategists consider state 
adopted “frameworks” as focal points for constructive public 
deliberation. 



Clarity of the NS F Reform Vision: The Content-Based 
Systemic Reform Model Applied in Urban Contexts 

The goal is to achieve wide spread support for expecting all 
students to perform at high levels. Teaching for and learning 
with understanding are behaviors described by the National 
Science Foundation's consultants for systemic reform, as 
"demonstrably effective pedagogical concepts... ‘inquiry-based’ ...which relieve 
students from the ultimately unproductive burden of rote learning” 
(Westat*McKenzie, October, 1998). This explanation characterizes the NSF’s 
position regarding pedagogy. Consistent with assumptions of Progressive educators, 
it posits development of critical thinking and problem-solving abilities as critical 
outcomes. At the same time, NSF embraced assumptions that “deep understanding 
of academic content” is a critical prerequisite for complex problem solving (O’Day 
and Smith, 1993, p. 262). Again, there are competing ideas about how curriculum 
and assessment standards should be set. 

America’s competitor industrialized nations sponsor national curricula that feature 
coherent, integrated, centralized policies. American education policies are set at the 
state level, resulting in the lack of coherent, centralized policies. The systemic 
reform model supports a unique common approach to state-by-state design of 
coordinated curriculum frameworks, standards, and assessments. Fuhrman, Elmore 
and Massell (1993) define policy integration, coordination and alignment towards a 
set of specific outcomes, as major features of systemic reform. With the USI 
initiative, NSF implicitly adopted the theory of alignment, using process and results 
"drivers,” as systemic change components to frame expectations for work in urban 
districts. Alignment of resources, policies, and behavior as a fundamental activity 
for systemic efforts, in theory, facilitates adoption and sustained use of standards- 
based curricula in science and mathematics. 

Consistent with emerging state systemic reform approaches, urban systemic reform 
focuses state reform expectations on urban school districts to achieve “a reduction in 
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The legitimacy and effectiveness 
of a systemic curricular approach 
will ultimately rest in the ability 
of the system to establish 
challenging curricular goals 
while striking a creative balance 
between the common culture and 
needs of the whole society on the 
one hand and the diverse 
perspectives, needs, and histories 
of subgroups and individuals on 
the other. 

(O’ Day and Smith, 1993 p. 294) 
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attainment differences between traditionally under served K-12 students and their 
peers" (Westat*McKenzie, September 1998, p. 10). The Council of Chief State 
School Officers (CCSSO) has invested in mechanisms to assist schools in 
monitoring implementation of curriculum and strategies designed to achieve the 
target reduction in attainment differences. The Survey of Enacted Curriculum 
(SEC) collects data on mathematics and science curriculum and teaching practices. 
SEC surveys teachers to determine what they taught, for how long, and with which 
methods (Blank and Kim, 2000). Analysis of survey results will document 
curriculum changes associated with systemic reform efforts at state and urban 
district levels. 

In addition to monitoring teacher reported changes in what is taught and how, NSF 
has commissioned research to determine the alignment of standards and 
assessments. The method of alignment study used at the state level will likely advise 
a similar process for reviewing alignment of urban district curricula with state 
standards. Researchers defined four alignment criteria to reflect the range of issues 
addressed by the standards. These include Content Focus, Articulation Across 
Grades and Ages, Equity and Fairness, Pedagogical Implications (Webb, 1999). 
Recently released results of a four state alignment study, indicate that the study 
design facilitated review and coding responses for only four of the five attributes of 
only one criterion — Content Focus. The researchers noted plans to review over 
time the fifth content attribute, Dispositional Consonance, defined as agreement in 
attention to students 5 attitudes and beliefs. Current study results do not reflect 
review of alignment of additional criteria, including Pedagogical Implications, with 
attributes related to pedagogy, such as engaging students and use of technology, 
materials, and tools (Webb, 1999). 



The Policy Driver : Backbone of Systemic Change 

Based on experience with State Systemic Initiatives, researchers have found that "in 
policy discussion, it frequently appears that reforms consist only of standards and 
assessments. Other central components, like support for classroom change through 
focused and enhanced teacher education and professional development, are less 
visible" (Fuhrman, 1995, p. 4-5). The most frequently heard message is that setting 
standards and measuring achievement will lead to better teaching and learning. Less 
attention has been paid to the formulation of policy to guide scaling-up from good 
examples (Hess, 1998). 

Researchers appear to believe that school funding and desegregation have generally 
been ignored as equity issues. An additive approach is in evidence; new practices 
and new policies are discussed, while old policies remain on the books (Fuhrman, 
1995). 
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Success has been reported for reform efforts that facilitate school-level networks that 
unite volunteer schools in collaborative efforts. Researchers report that networking, 
as a concept, holds promise, because membership promotes access to knowledge for 
change. Knowledge about change and public information about the challenges of 
reform have been useful in building necessary public support (Fuhrman, 1995). 
Public support for sustained, long-term reform activity is important, as research 
indicates that pressing for immediate results may not provide the best conditions for 
implementing desired change in volatile urban environments (Hess, 1998). 

Additional research (Hirth, 1996; Fuhrman, 1995) indicates that systemic reform is 
fundamentally related to state school finance policy changes. Effective reform will 
be evidenced in specific allocations of resources for institutionalizing change and 
should clearly define how available resources are realigned to accomplish change. 
Perhaps, most significantly, there should be evidence of efforts to achieve greater 
equity in state support of urban schools where students and communities have more 
need. 

A study by Grissmer, Flanagan and Williamson (1998) suggests that neglect of 
school finance policy reform might constitute the "Achilles heel" of systemic reform 
efforts, in light of empirical data suggesting that funding to schools and 
disadvantaged families influences educational outcomes. NAEP results indicate 
score increases of disadvantaged minorities and resource increases targeted to that 
population from the 1960's to the early 1990's. 

• Evidence that effective increases in real resources from 1967-1991 available 
to boost achievement of regular students has been markedly overestimated 
due to failure to correct for inflation and the diversion of resources to 
students with disabilities [in previous research] 

• Evidence that the more limited real resources available to increase 
achievement scores from the late 1960's to the early 1990's was 
disproportionately targeted at minority and lower income children 

• Evidence that minority and less advantaged children have made substantial 
gains in test scores in the 1970 to 1990 period, while more advantaged 
students have made only small gains 

• Evidence that the timing of score gains of minority children seem to be 
related to both the civil rights and war on poverty efforts as well as declines 
in class size (Grissmer, Flanagan, and Williamson, 1998, p. 2). 

Policy also concerns state and local alignment of "what is taught" with "what is 
assessed", with "how teachers are prepared and supported," as well as with "how 
teachers are evaluated." NSF's approach to systemic reform reflects broad 
agreement on what should constitute curriculum (content standards), and it avoids 
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constricting local options for how alignment is achieved. This approach encourages 
diverse communities to focus on local realities (Knapp, 1997; Fuhrman, 1995; 
Westat*McKenzie, September, 1998). 

Data collected for this study, from each affected district, then, reasonably seek 
evidence that local school boards and communities are addressing eight questions 
related to policy alignment: 

1. To what extent has the local district adopted national/state mathematics and 
science standards? 

2. How are state and local systems insuring desegregated, equity-based access 
to high quality science and mathematics teaching and learning? 

3. What local district policy changes indicate the shift to standards-aligned 
practices in student evaluation, teacher selection, and teacher support? 

4. What local district policy guides school level voluntary networks for 
professional development and skills sharing? 

5. What local district policy changes indicate the shift to standards-aligned 
practices in text book selection and other curriculum shaping activities, 
including use of informal and enrichment learning resources, such as science 
centers, zoos, etc.? 

6. What local district policy changes indicate new or stronger relationships 
with university programs for teacher preparation and recruitment? 

7. What policy additions or changes support development of major advocacy 
and public information relationships with key local and state stakeholders? 

8. What policy changes indicate a monitoring approach that continues during a 
time period that realistically allows the system to embrace genuine change. 



The Outcome Drivers: Where Policy Meets Implementation Reality 

The object of change is improved student achievement and competitive student 
outcomes in urban schools. While inevitably this study must view by individual 
district achievements, based on the common set of variables associated with the six 
NSF change drivers, the challenge of the current study is to go beyond case by case 
descriptions of reform implementations in states and urban districts (Knapp, 1997). 
This study attempts documentation of specific overarching common variables that 
indicate movement towards the teaching and learning reforms that hypothetically 
contribute to desired student achievement outcomes. The systemic reform initiative 
challenged school districts and communities to research and conceptualize 
(envision) different content and different ways of interacting with content, to craft 
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policies that promise to guide long-term improvements in system supports for the 
new vision, and to fully implement change. The study of each community’s 
response requires close inspection of policy, its translation in day-to-day practice, 
and student outcomes defined by quantitative and qualitative indictors. 



[The reform vision] is one that 
emphasizes the learner's understanding oj 
central ideas and processes; application 
of ideas and skills to non-routine, 
complex problems; in-depth immersion in 
important themes and topics, rather than 
exhaustive coverage of materials; active 
mental (and often physical) engagement 
with scientific phenomena; exposure to 
authentic, real-world phenomena, 
problems, and scientific activities 
(Knapp, 1997, p. 6; Thompson, Spillane, 

& Cohen, 1994 as referenced by Knapp). 



Knapp (1997) characterizes NSF's vision for 
teaching and learning as "elaborate and 
ambitious,” and he delineates critical realities 
from which to view implementation progress. 
These critical realities are discussed below, as 
they also surface in the research and reports of 
science and mathematics educators, as well as 
in the studies and policy analyses reported by 
urban education experts, as underlying factors 
in viewing success and progress in urban 
school reform implementation. 



EVOLVING CHAIN OF INFLUENCE IN SYSTEMIC REFORM 



Changing Ideas about Science , Mathematics y Pedagogy , and Learners 

A report released in 1998 by the U.S. Department of 
Education analyzes data from the Third International 
Mathematics and Science Study (TIMSS). 

Researchers conclude that the relatively poor 
performance of U.S. students on international 
mathematics and science achievement tests are related 
to differences in curriculum (content) and pedagogy 
(process) between U.S. schools and schools in 
competing nations, such as Japan and Germany. 

Other curriculum specialists, however, question the 
utility of TIMSS comparisons specifically those 
related to curriculum, because the TIMSS data 
generalizes to the American system, and it does not 
account for varying results, given the absence of a 
national U.S. curriculum (Usiskin, 1998). Usiskin 
(1998) also questions studies of instructional 
approaches, which he believes yield little that can be 
replicated. Instead, he favors studies of materials, 
such as textbooks, and definitive studies of student 
performance when NCTM endorsed curriculum 
(standards) and problem-solving approaches 
(pedagogy) are used. 

Interestingly, standards-based teaching as defined by 
Usiskin (1998) advocates instruction that fosters 
knowledge of underlying mathematical concepts and 
the ability to apply concepts. Usiskin appears to 
believe that critical problem solving ability will result 
from use of specific curriculum materials and 
University of Chicago School Mathematics Project 
(UCSMP) texts, published by Prentice Hall. 

In contrast to NCTM's support for a single or defined text adoption, the School 
Science and Mathematics Association (SSMA) takes a different approach to identify 



Among the TIMSS study's major 

findings are the following: 

• U.S. schools compare poorly 
by doing less with more 

• Science and mathematics 
curricula and textbooks are too 
diffused, unfocused , and not 
challenging. 

• American courses include more 
topics covered and less time is 
spent on each topic, resulting 
in reteaching topics, but 
providing too little of any one 
thing. 

• The mathematics curriculum 
fosters less thought, less 
achievement. American 
schools teach procedures, not 
understanding. 

• U.S. teachers rarely develop 
mathematical concepts in 
instruction. 

(Forgione & Gonzales , 1998, 
p. 6-19) 
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major curriculum tools. SSMA supported development of an instructional materials 
evaluation system, in relationship to Virginia's State Systemic Reform effort. 
Representatives from eleven states guided development of thirteen (13) criteria for 
selection of instructional materials in science and mathematics, including Analytic 
and Likert instruments to measure the criteria (Koontz, 1996). 

In other research, texts and instrumentation are subordinate 
concerns. As noted by Usiskin (1998), much mathematics 
(science) education research is concerned with instructional 
techniques (p. 9). Metz (1995) examined children’s ability to 
engage the inquiry process. She concluded that meaning and 
application ability can be fostered if students have early and 
frequent opportunities to pose questions, gather and interpret 
data, and revise their own theories. 

Inquiry, as a primary teaching/leaming approach, is a 
significant component of NSF's implicit theory of standards- 
based reform, and it signals the importance of orienting 
thinking about science, mathematics, and pedagogy to make 
inquiry learning opportunities available to a range of learners. 
NSF's advocacy for every school's use of state and national 
standards to create "high quality" teaching/leaming environments is a central 
expectation of the Urban Systemic Initiative. 



Such a learning environment... 
strive[s] for universal acceptance 
and classroom implementation of 
such demonstrably effective 
pedagogical concepts as \ hands- 
on ' and 'inquiry -based' education 
which relieve students from the 
ultimately unproductive burden oj 
rote learning ( Westat*McKenzie , 
October 1998, p.5). 



Assembling Pieces of a Complex Puzzle 

Educational research provides a rich backdrop for engaging 
pedagogical change. There might, perhaps, exist so much 
theory and recommended strategy that the most skillful reform 
leaders will first own a personal commitment to a theoretical 
framework. Still more effective are those frameworks that 
provide concrete examples of how they work. Adoption of 
inquiry as a systematic instructional approach adds a number 
of “pieces” to an already complex set of instructional 
considerations. 

Different ideological frameworks are implicated in various and 
even similar instructional approaches. Constructivists' ideas 
are central to advocacy for investigation, as a pedagogical 
approach. Constructivists' ideas follow both cognitive and 
social orientations. Cognitivists are most concerned with 



Only by providing schools and 
parents with a choice of valid, 
well-supported models of 
schooling can we make the 
change process manageable for 
school personnel, and satisfy the 
conflicting demands of the public. 
If we do not create a gallery of 
clear models, the individual 
puzzles will remain blurred and 
grow increasingly fractured 
(Doss, 1998, p. 594). 
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learners’ interactions with the physical world— manipulation of objects and hands-on 
experiences. The content-based standards-derived inquiry advocated by the science 
and mathematics organizations responsible for developing the content standards is 
based on Cognitive Constructivist ideas. (AAAS, 1989; NCTM, 1989) 

Social Constructivists believe that understanding and learning occur through socially 
mediated events and that social interaction, the cultural context, and language use 
are fundamental to thought. ’’Educators and researchers ... are interested in how 
students think about scientific problems and how they use their own understandings 
of scientific processes to build conceptual models." For these researchers discourse 
in science classrooms indicates what is learned (Richmond and Striley, 1996, p. 

839). Science/mathematics literacy, according to this line of reasoning, is reflected 
when students have the ability to develop problem-solving tools, including 
observation, reflection, and assimilation of knowledge into their own models or 
theories about phenomena in the physical world. This ownership of understanding 
or meaning-making is believed to contribute significantly to student performance, 
and it poses additional issues in systemic reform for urban schools, given the 
cultural and linguistic diversity of students in those schools (Garcia & Gonzalez, 
1995). 



Students' attitudes are also believed to influence 
achievement results, and might be considered the "buy- 
in" factor which is associated with perceptions of 
efficacy (Greenfield, 1996). The work to "hook" 
students on science involves "incorporating students’ 
culture and language in science teaching, orientation of 
science pedagogy toward reciprocal interaction, and 
teacher behavior that empowers student curiosity 
(Atwater, 1996, p. 831). Students "buy-in" when 
teachers trust and challenge them to find answers, and 
relate information to community or societal realities. 

When instruction requires students to ask and answer 
questions, teachers incubate scientific habits of mind. 

Students "buy-in” when they are interested in what 
they study and perceive knowledge to have utility in 
their life planning. Sociology studies describe urban 
student motivation in research documenting places of 
hope for inner-city youth, where students welcome 
responsibility and enthusiastically engage in activities 
that support their escape from inner-city despair and 
reshape their life chances (Delgado-Gaitan, 1993; 

McCaleb, 1994; McLaughlin, Irby, & Langman, 1994; Wehlage et al., 1989). 



...if appropriate practices are 
consistent over a long period of time, 
children can and do learn to enjoy 
and value mathematics. There is a 
building body of evidence that 
indicates the larger, more general 
goals of schooling can be 
restructured and reinvented with a 
fair degree of success so that the 
school culture becomes conducive for 
student learning and motivation. 

... evidence that classroom practice 
can be positively reinvented so that 
the culture of the classroom can 
become conducive for learning and 
enjoying mathematics. 

(Maehr & Anderman, 1993, in 
Middleton & Spanias, 1999, p. 85-87) 
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An analysis of research on student achievement motivation, reported by Middleton 
and Spanias (1999), finds that "achievement motivation in mathematics [science] is 
highly influenced by instructional practices" (p.85). 

The concept of best practices acknowledges the interrelationship of social, cultural, 
historical and psychological factors. Schools function in communities, and effective 
practices are based on models and research linking teaching and learning to 
community realities. “The situation-specific nature of the kind of teaching and 
learning envisioned by school reformers is the key challenge for teachers' 
professional development. ..success. ..ultimately turns on teachers' success in 
...learning the skills and perspectives assumed by the new visions of practice and 
unlearning the practices and beliefs about instruction that have dominated their 
professional lives to date. ...teaching for understanding relies on teachers' abilities to 
see complex subject matter from the perspectives of diverse students” (Darling- 
Hammond, 1995, p. 597). 

Best practices engage students (Hilliard, 1989). Students are more successful when 
they are more engaged. Students are best engaged when their teachers and schools 
see the education enterprise from the perspective of the students and their parents, 
implicating the need for “new teacher perspectives (Keith, 1995; Liebling, 1996).” 
The reform agenda, therefore, must include new active local community 
relationships that cast science and mathematics learning as important issues, 
especially for resource poor communities (Ramsey, 1995). 

Standards-informed, emergent ideas about teaching and learning mathematics and 
science suggest several systemic reform challenges: changing student attitudes about 
science, changing teachers' perceptions of what works in teaching science, and 
changing teachers' perceptions of who can be successful in performing in open- 
ended investigatory tasks. Outcomes indicators which serve as validation measures 
for reform, according to researchers, will make school success in science, 
mathematics, and technology curricula a key criterion. Research suggests that 
effective curricula are accessible and useful in the lives of students with the "range 
of abilities and dispositions that a robust democratic society requires.” Effective 
curricula lead to “circumstances of richly textured educational environments guided 
by caring, competent, and professional teachers" (Goodlad, 1998, p. 670; Darling- 
Hammond, 1994; Kyle, 1998; Wiggins & McTighe, 1998). 



System Requirements and Regulators of Practice: 

Defining "High Quality" is Important 

Defining "high quality," by leveling appropriate expectations for performance, 
appears a critical challenge. Performance expectations are visible in assessment 
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measures, and assessment, therefore, must be examined in light of some 
conceptualization of the teaching and learning behavior that constitutes "high 
quality." If, as NSF documents suggest, "high quality," means that "learning is a 
process in which youngsters are active participants in the acquisition and 
construction of knowledge" (Westat*McKenzie, September 1998, p. 9), then 
reasonable assessment modes will reflect students and learning as active and 
interactive agents (Kyle, 1997). Otherwise, imposing traditional standardized 
performance measures "obstructs efforts to improve science education," and runs 
contrary to objectives for engaging students in rigorous intellectual activities that 
foster meaning and contribute to understanding (Kyle, 1997). 

Meeting standards for high quality, then, might occur if systems develop 
mechanisms for viewing or measuring success in relationship to authentic 
indications of what students can do-displays of outstanding student work which 
have been noted as "oddly missing from most American schools and from most of 
the arguments about how to repair them" (Cohen, 1995, p. 751). Without an 
understanding of the range of results possible from an instructional program, neither 
consumers (parents, students, community stake-holders) nor regulators (school 
boards, school personnel) can develop practical/transforming behavior or 
expectations for results, beyond standardized test scores (Cohen, 1995 p.751). 

Adoption of appropriate assessment measures is dependent on development of such 
measures. Researchers assert that scientific [mathematical] inquiry has resisted 
analysis, but studies by Cognitive psychologists and science educators should result 
in progress in defining instruments and approaches (Germann, Aram, Odom, and 
Burke, 1996). Several lines of reasoning will influence development of assessment 
protocols. 

Cognitivists view the key skill in science investigation as the ability to distinguish 
and coordinate the relationship between theory and evidence. Researchers have 
determined that young children do not differentiate an explanation from data or 
evidence that might support or refute the explanation. These studies indicate that 
children's pnor knowledge significantly influences hypothesis formulation and 
experiment design (Kuhn et al., 1988 in Germann et al., 1996). 

Other Cognitivists consider prior knowledge an essential link for adding new 
knowledge. In this research, process is emphasized and requires activation of long 
term memory (prior knowledge) to represent the problem, search for remembered 
knowledge as the model for fashioning solutions and evaluating the chosen solution 
(Klahr & Dunbar, 1988 in Germann et al., 1996). 

Science education research has focused on the development of process skills, 
including posing questions, hypothesis formulation, variable identification and 
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description, experiment design, data collection, manipulation and analysis. Science 
educators believe that practical involvement and skills use result in increased 
proficiency, and conclude that assessment should be consistent with instruction and 
learning (Germann et ah, 1996). Research in this area has shown that students have 
inadequate experience for proficient performance on assessments of laboratory 
process skills (Germann et al., 1996). The assessment protocols associated with this 
research do not discount open experimental inquiries where students pursue their 
own causal questions and pursue independent reasoning modes. While these more 
authentic protocols are useful in determining process mastery, they pose special 
challenges for assessing knowledge in relationship to content standards (what 
students should know), which have been traditionally measured by standardized 
paper and pencil tests. 

Use of standardized tests, as significant evidence of school quality and reform 
progress, and public scrutiny of their results force school personnel into defensive 
positions. Principals and teachers perceive threats to their livelihood, and 
antagonistic political pressure can ’’derail" reform efforts (Darling-Hammond, 1994). 

Tests ought to measure what matters, and many assessment researchers are not 
advocates of high-stakes testing. Baker, Glaser, & Jones (1999) agree that the more 
we keep students, the more they achieve— the more we engage them, the more we 
keep them. According to these researchers, there should be measures of what 
matters, beginning with engagement. We have not yet adopted this insight in local, 
state, or national assessment approaches. We lack instrumentation, or perhaps we 
have not acknowledged instruments at hand for the right purposes. Standardized test 
misuse is a major threat to the success of systemic reform implementation (Cuban, 
1991). 

A number of pedagogy, policy, and science education researchers assert that 
meaningful and positive change will result when principals and teachers are allowed 
to de-emphasize testing and lead from the trenches, using "best practices" to create 
domino-like examples of student success. The most powerful regulator of practice is 
the quality of the dynamic between principals and teachers (Cuban, 1989). School 
system policy and practice frame the interaction, and in the current reform 
environment, decentralized decision-making facilitates the authority of principals 
and teachers to shape implementation. Beliefs guide human interactions, and unless 
professional development efforts are effective in shaping mind-sets for "untracking 
science education," traditional mind-sets about who can do science will prevail 
(Kyle, 1998; Wells & Oaks, 1996). 

In decentralized decision-making environments, vision and influence from the top 
levels of the system are particularly important. The rapid turnover of 
superintendents in urban schools works against development of stable, appropriate, 
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systemic mind-sets to support reform implementation (Hess, 1998). Reform 
implementation appears most advantaged when goals of movement are clear in the 
minds of key decision-makers. Communities attempting to reform schools would do 
well to support superintendents who understand authentic achievement and will risk 
establishing system-based evaluation criteria that assists principals and teachers in 
organizing instruction to allow students maximum opportunities to produce 
knowledge, create things, and demonstrate understanding (Education and Urban, 
Society [editorial], 1992). The highly charged political environments surrounding 
most superintendents, given the often adversarial relationship of school boards with 
their system leaders does not support appropriate goal-setting and risk-taking 
necessary to reorient thinking and engage in practical behaviors that engender 
student success (McAdams, 1997; Darling-Hammond, 1994). 



Formation of Coalitions of Actors and Stakeholders 

Urban stakeholders need tools that facilitate thinking together about how to improve 
ineffective systems that are not equipping students with literacy and thinking skills 
(Darling-Hammond, 1995; Fullan, 1996; Goodlad, 1998). Despite questions of 
whether merely setting standards and measuring achievement can improve teaching 
and learning, doubters and reform supporters, alike, can acknowledge the utility of 
standards-forced problem-solving, with a clear view of achievement levels which, if 
reached, improve hope and opportunity. 

Standards, then, allow parents and other community constituents of school boards, to 
take stock and require an alignment of resources. Research suggests that success of 
reform implementation is related to "local context and the vision of change agents," 
as well as that of those who emerge as leading stakeholders, specifically in 
community and political roles (Zetlin & Lim, 1998, p. 516). It is important for 
political agents to do more than make noise that benefits broader political agendas. 
Effective reform advocacy for constructive change requires understandings that are 
shared among the professional education community, local community-based social 
and political institutions, as well as the larger economic and technical employment 
community (Tyack & Cuban, 1995; St. John & Pratt, 1997). 

Some research advocates reform that creates an environment where stakeholders 
demand human, physical, curriculum resources, as well as system accountability for 
transforming urban schools into high achievement, success-focused environments, 
providing "high quality, rigorous mathematics and science education to all students" 
(Westat*McKenzie, October 1998, p. 7). 
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Forms and Availability of Supports for Instruction and Instructional Change 



Teachers work directly with students , so 
discussions of capacity often focus on what 
teachers need to know and be able to do. 

Early discussions of the knowledge base for 
teaching addressed both propositional 
knowledge and procedural knowledge or skills. 
Consequently, discussions of staff development 
dealt with the content to be learned in 
workshops and the methods of skills and 
training . Theorists subdivided knowledge into 
areas such as knowledge of subject matter, 
knowledge of curriculum, knowledge about 
students, and knowledge about general and 
subject-specific pedagogy . Recognizing that 
teachers might know how to teach in a 
particular way yet still not do so, researchers 
have called attention to other areas of teacher 
capacity . In order to enact reform, for 
example, teachers must have the disposition to 
meet new standards for student learning. 
Studies of teachers' attempts to change their 
practice suggest that the capacity to teach in 
new ways is also connected to teachers' views 
of self to their beliefs about their role in 
classroom activity, and to the personality they 
adopt in the classroom. Also critical are 
teachers' views of themselves as learners, 
including what, where, and how they will 
learn. Thus building teachers' capacity to 
enact reform means more than helping them 
gain additional [subject] understanding and 
[pedagogical] skill; it also means encouraging 
them to alter the goals they hold for their 
students and to change the way they see 
themselves. 

(Floden, Goertz & O'Day, 1995, p. 21). 



Glidden (1999) described schools in low- 
income communities with above national 
average schooling results, including test 
scores, attendance, grades and completion 
rates. He attributes differences in the success 
indicators of demographically similar schools 
to differences in school building cultures. 

More successful schools have more engaged 
school cultures and climates. 

Building engaged culture is related to the 
complex issue of developing reform capacity. 
As researchers Floden, Goertz & O'Day (1995) 
write, even when the vision is clear, policy and 
goals are aligned and teachers are willing, the 
system lacks the critical knowledge, capacity 
and resources for effective reform 
implementation. 

Teachers' perceptions are important, and a 
major issue in reform implementation may 
emerge as the disjunction between beliefs 
about successful urban teaching held by 
professional reformers (university subject area 
and pedagogy specialists) and experienced 
teachers. Ilmer, Snyder, Erbaugh, & Kurz 
(1997) found significant difference between 
the views of university teacher educators and 
experienced teachers about the importance of 
responding first to "children's needs" as a 
factor associated with successful teaching in 
urban schools. Teachers were found to view 
responding to children’s needs as more 
important than university educators did. The 
concern for caring culture has, perhaps, been 
overlooked by science/math education 
reformers, although it is difficult to ignore the 
large body of literature related to urban 
schooling, in the context of USI 
implementation concerns. 
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Culture as a Factor in Urban School Reform 

Several factors associated with culture appear relevant to discussions of reform 
implementation in urban schools (Hollins, 1999; Chin, 1996; Ladson-Billings, 1994; 
Foster, 1993; Irvine, 1990; Cochran-Smith, 1997). Salient views emerging from this 
literature cohere with those of the science/mathematics education reform 
community, along the following dimensions: 

1. Teachers and their advocates do not accept low achievement or failure to 
produce; 

2. Teachers and their advocates are highly involved in planning for students 
with each other and with students; 

3. Teachers link students to external resources and experiences; 

4. Teachers lead and drive planning; 

5. Teachers control some resources; 

6. Teachers and students interact in defined small groups; 

7. Students care about each other and value belonging to the school group; 

8. Students connect their academic work with larger purposes and with 
achievements that they target for themselves; 

9. Students have responsibility for achievement; 

10. The principal is an optimistic facilitator, working to support the achievement 
defined by students and their teachers. 
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TEACHER TRAINING AND PROFESSIONAL DEVELOPMENT 



Teachers ' Perceptions of Change Capacity Influence Change Potential 

Teachers' perceptions of whether internal and external supports are appropriate and 
helpful appear related to levels of change reported in teaching/leaming behavior. 
School-building level instructional design teams, led by former teachers who have 
enacted preferred instructional models are influential. There is some tension between 
the understood value of teachers owning, by creating, their own instructional models 
and the teachers' concern that they need concrete examples of models that work 
(Bol, Nunnery, Lowther & Dietrich, 1998). This tension is likely an artifact of the 
challenge for building capacity that is neutralized as teachers own membership in 
supportive professional community. 

Teacher development in a supportive professional community appears an essential 
system capacity, in evidence in environments where student understanding and 
success define the purpose of schools (Floden, Goertz, & O’Day, 1995). Hollins 
(1999) describes a process for helping teachers develop habits of mind that support 
student achievement. According to this work, student achievement is positively 
influenced when teachers and administrators talk and think in positive ways about 
the educational needs of their students. Students learn when teachers are active 
problem-solvers, engaged with their students and each other in figuring-out 
curriculum. 



Effective Professional Networking Associated with Student Achievement 

In research designed to connect two significant longitudinal studies, the National 
Educational Longitudinal Study (NELS:88) and the 1984 High School and Beyond 
teacher survey, researchers relate teachers' assessments and perceptions of students' 
abilities to amount of collegiality and level of involvement in professional learning 
communities (McLaughlin and Talbert, 1993). Teachers with more collegial support 
have more positive views of their students' potential. Teachers who are engaged 
with colleagues are most able to adjust practice to facilitate student learning or 
success. 

Research appears to support the importance of teachers working with other teachers, 
to solve problems related to the achievement of their students. Teachers benefit 
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from active relationships with other professionals in industry and higher education. 
Such associations form effective professional development networks. The teachers 
and their professional development partners are most effective when they view 
deficits in student performance as problems that are resolvable, given use of 
appropriate strategies and access to resources. Students achieve more success when 
teachers work as problem-solvers and supportive colleagues within the school 
building and within a professional development network. 

It appears that a key reform implementation challenge is alignment of expectations 
for system accountability with long-range capacity-building behavior. Falk and Ort 
(1998) warn that unless teachers have opportunities to work with standards in ways 
that allow them to balance their individual views (or perhaps in the case of 
experienced urban teachers a collective view of what constitutes best strategies for 
working with their students), standards-based reform is "toothless." Of equal 
concern are state and local school district approaches that penalize teachers by 
holding constant expectations, using standards rhetoric, without learning about 
students and their work. Reform implementation conducted in this spirit undermines 
essential teacher decision-making and professional insight. Research suggests that 
teachers interacting with other teachers, students, students' families and communities 
about authentic student work is critical engagement for supporting desired change 
(Falk & Ort, 1998). Taken together, engaging, networking, reculturing and 
restructuring appear to provide critical underpinning for long-term positive school 
change (Fullan, 1996). 



Attracting , Employing and Retaining Effective Mathematics and Science 
Teachers : Supporting High Achievement through Course-Taking Behavior 

The content backgrounds of teachers influence school capacity to offer courses in 
the schooling sequence associated with competitive student performance. When the 
mathematics achievement scores of 13 year old American and Japanese students 
were compared in 1992, the aggregate score of American students was 6.4 points 
lower than the aggregate score of the Japanese students. When disaggregated by 
type of mathematics course taken, the score of 1 3 year old Americans who had taken 
algebra was 10.6 points higher than the aggregate score of 13 year old Japanese (U. 
S. Department of Education (1993). To provide access to courses associated with 
higher achievement, school districts must have teachers who are qualified to teach 
those courses. Goldhaber & Brewer (1998) found evidence that subject specific 
training for math and science teachers has a statistically significant positive impact 
on student achievement, when achievement is measured by standardized tests. 

The predicted magnitude of the effect of teacher training on student achievement is 
relatively small. In science, the mean 10th grade test score was about 22 with a 
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standard deviation of 7.5. Having a science teacher with a bachelor's degree in his 
or her subject is predicted to add about .7 points to the average science student's 
score relative to what the student would have received had the teacher not had a 
bachelor's degree in science. Thus a science-specific bachelor's degree is worth 
about one-tenth of a standard deviation improvement on the 10th grade science test. 
[In mathematics]. ..again, the improvement is relatively small. The difference in 
predicted scores for a student whose teacher has a bachelor's degree (not in math) 
and one whose teacher has a bachelor’s degree and a master's degree in math is about 
1 .4 points. This is only one-tenth of a standard deviation on the 10th grade 
mathematics test (Goldhaber & Brewer, 1998, p.134). 

In the same study, these researchers confirmed previous findings that 75% of 
achievement, when measured by tests, is explained by individual and family 
backgrounds, such as level of parents’ education. They conclude that to the extent 

school districts are interested in improving 
student performance on standardized measures, 
those districts will pursue employment of 
appropriate reward mechanisms to keep math 
and science teachers with degrees in those 
disciplines. This research, and other work, 
found primarily in work relating economics and 
education, find little relationship between 
schooling inputs, such as class size and teacher 
pay, on student achievement (Card & Krueger, 
1996; Ehrenberg, 1994; Ehrenberg & Brewer, 
1995). The challenge, it might appear, is to 
attract and retain professionals with 
mathematics and science degrees for teaching 
positions in high poverty schools. 

Additionally, findings of researchers on the 
impact of the Chapter 1 (a federal compensatory 
program, a.k.a. Title I) are germane to questions of predictive and correlative 
variables associated with academic achievement in high poverty schools. Using 
standardized test measures, this research also indicates that students in high poverty 
schools are far less likely to master basic reading and mathematics skills than are 
their counterparts in lower poverty schools, with a more pronounced performance 
gap in higher order thinking tasks (Puma, Karweit, Price, Ricciuti, Thompson & 
Vanden-Kieman, 1997, in McClure, 1997). Further, these researchers found that 
students receiving specialized, "pull-out" services scored lowest when they received 
more intensive services, and that over time, the services did not "close the 
performance gap between economically disadvantaged students and their more 
advantaged peers” (McClure, 1997, p. 342). 



If, however, 75% of achievement can be 
explained by socio-economic and personal 
variables, what is the systemic change power of 
the reportedly "small" positive differences 
attributed to the teachers subject matter 
preparation or, for that matter, any of the other 
reform driver variables? This research forces 
an unresolved quandary for those seeking 
predictive, mathematics-based models for 
explaining student achievement in urban 
communities, which are characterized by 
concentrations of poverty, particularly if 
standardized achievement measures are the 
predominant regulators of system practices. 
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The research also found that authentic classroom work samples which rated top 
grades in high poverty schools were rated average and below in low poverty schools. 
Additionally, access to a large national, longitudinal database allowed the research 
to view a range of factors and their influence on standardized test measures. A 
review of the findings yields the following observations: 



One of the more interesting findings 
of this research relates indirectly, but 
perhaps significantly, to the work of 
Goldhaber & Brewer (1998). Puma et 
al. (1997; in McClure, 1997) found 
that in the highest poverty, lowest 
achievement schools services were 
provided by the least well credentialed 
(paraprofessionals or aides) personnel. 
Although Goldhaber & Brewer (1998) 
argue that certification and degrees in 
other than math/science content have 
insignificant impact on test-score- 
measured student achievement, their 
work taken with the McClure (1997) 
report of Puma et al. (1997) findings, 
appears to strengthen the case for 
professional development, as a key 
factor in efforts to improve school 
outcomes. 



Additional research reported by McClure (1997) indicates that when researchers 
performed a disaggregated analysis of Chapter 1 data, they found more positive 
results with the more economically disadvantaged students and greater reduction in 
the performance gap between more and less impoverished students associated with 
active collaboration and coordination of curriculum materials between Chapter 1 and 
regular classroom teachers (D’Agostino, Wong, and Hedges {in press}, as reported 
by McClure, 1997). Research findings, again, implicate the centrality of effective 
teacher behavior in changing student outcomes, with urban, poor students. Given 
that we are "stuck" with standardized measures until we evolve effective formative 
and summative authentic approaches, schools, then, are challenged to attract well 
prepared teachers and provide environments that foster their effectiveness. 

The literature suggests that effective teachers work in environments where the ethos 
and culture of the teaching community support problem-solving discussion and 



... based on a longitudinal sample. ..some factors. ..had 
strong , positive, and consistent relationships to 
improved student outcomes for disadvantaged 
learners. Among student and family characteristics, 

... parents' expectations for better than average 
grades and future college attendance were positively 
and significantly related to scores on standardized 
achievement tests and teacher ratings. School factors 
were also associated with better student outcomes; 
those with lower concentrations of poverty 
(independent of the economic status of an individual) 
produced higher standardized achievement test 
scores. Other school characteristics positively related 
to student achievement were: avoiding grade 
retention (retained students had lower test scores than 
their classmates) , an instructional emphasis on 
comprehension and the development of and 
appreciation for the ability to write, lower levels of 
student disciplinary actions, and household stability 
in the early grades. (McClure, 1997, p.342). 
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behavior among colleagues, supported by a professional development community of 
other experts (Hollins, 1999; Glidden, 1999; Swanson, 1995; Gold & Roth, 1993; 
Goodlad, 1993; Weisbender, Champaign, & Maddahian, 1989; Veenman, 1984). 
When these conditions exist, change or measurement by external standards is more 
easily accommodated, and experts believe that use of new standards and high stakes 
testing are less intimidating. 

Instead of adhering unquestioningly to artificial external pressure and failing many 
students, or resorting to practicing for the test, effective schools encourage hands-on, 
minds-on problem solving (Kyle, 1997 ; Bradley, 1991). In these environments, 
teachers assume empowerment to nurture understanding, insight, and excitement. 
The school community— all stakeholders— celebrates authentic demonstrations of 
student achievement (Darling-Hammond, 1995; Siddle-Walker, 1999). 
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CONCLUSION 



Summary of Theoretical Chain of Influence Leading 
from Policy to Classroom Practice to Student Performance 

Policy alignment under girds the notion of systemic change. When the six NSF 
driver variables are analyzed with guidance provided by the framework offered by 
Knapp's (1997) "avenues," critical relationships emerge between policy and 
systemic behavior. The review of research studies reveals little or no researcher 
agreement regarding variables to be used as criteria in model building. 

There is general agreement that teacher variables and thus professional development 
are key to student success; however, there is little agreement about what constitutes 
appropriate professional development. In some instances it is viewed as improved 
teacher content knowledge, elsewhere it is improved pedagogical approaches for 
presenting content, while other researchers view professional development activities 
as improved cultural immersion on the part of teachers. Regardless of how it is 
defined, professional development is a significant variable in research investigating 
systemic change. 

Additionally, researchers rather universally agree that curriculum is a central issue in 
improving student performance. During the last decade this seems to have evolved 
under the term “standards-based curriculum.” Teaching and learning capacity in 
standards-informed environments is a reflection of the quality of leadership provided 
for and problem solving directed towards teacher training and development. Thus 
professional dynamics that influence curriculum should be a central issue in any 
model building. 

Finally, student performance variables, while pivotal to model building suffer from 
the vague agreement about what performance measures constitute appropriate 
indices. Certainly, student knowledge utilizing some standardized measure should 
be one component of any model. This could vary from state-mandated tests to 
standardized measures such as the CAT, ITBS, CTBS-4, etc. Measures of 
qualitative indicators such as student and teacher engagement are important to equity 
concerns. 

There is little guidance in the research literature to inform construction of a causal 
model beyond the global areas enumerated above. The complexity of the reform 
endeavor derives from the highly interactive nature of the drivers. The current 
research distills the relationships to interactions in three key strands— policy, 
professional dynamism, and assessment. 
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APPENDIX A: 



USI EVALUATIVE STUDY ABSTRACT 

The purpose of the study is to determine the impact of the National Science 
Foundation’s Urban Systemic Initiative (USI) program on student achievement and 
the learning infrastructure in urban school districts, and to develop an inferential 
causal model that relates the SI drivers and other key elements to the outcomes 
observed. Our study team will analyze the effectiveness of the USI program from 
five different perspectives: across all 21 USI sites; by USI cohort; longitudinally 
within sites; by comparison with 7 non-USI cities; and within a national context that 
includes other major reform efforts. 

The study team will develop a Key Indicator Data System (KIDS) to collect/compile 
annual core data from the baseline year using both quantitative (K-l) and qualitative 
templates (K-2). Our study team will conduct an annual KIDS workshop designed 
for USI core data managers and evaluators to enhance data integrity and share 
expertise. Based on collaborative research agreements, we will receive SAT-I, AP, 
and ACT test results (as part of K-l data) directly from the Educational Testing 
Service, the College Board, and ACT, Inc., respectively, for the next three years. 

Each year, the team will conduct four site visits including interviews with USI 
leaders, focus groups, and school visits, to further explore detailed implementation 
issues. In addition, a modified version of the Council of Chief State School 
Officers’ Survey of the Enacted Curriculum will be administered to 80 selected math 
and science teachers within these four sites. Finally, we will gather and review 
information from national databases such as the Third International Mathematics 
and Science Survey (TIMSS), National Assessment of Educational Progress 
(NAEP), Goals 2000, Equity 2000, as well as the USI sites’ annual reports and 
relevant documents. 

The team has identified key research questions in eight areas mapped to the NSF 
drivers: (1) student outcomes, (2) curriculum and instruction, (3) assessment 
instruments, (4) professional development, (5) policy, (6) leadership, governance, 
and management, (7) convergence of resources, and (8) broad-based support for 
reform. We will use a variety of statistical analysis methods to validate the 
hypothesis that a well-implemented USI program has a positive impact on student 
outcomes. Further systemic analysis will explore the determinants of successful 
implementation of urban reform in mathematics, science, and technology education. 
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During the three-year study period of October 1998 to September 2001, we will 
produce a USI Annual Fact Book and USI Evaluative Study Report, both in print 
and in CD-ROM format, and a master data base to be available on the world- wide 
web. Beginning in Year 2 we will publish a series of newsletters and monographs 
detailing our research findings. 

Our research team has expertise in systemic education reform, assessment, 
evaluation, academic research, and core data collection and analysis. A research 
team of three co-Principal Investigators, all of whom have been involved in the NSF 
systemic reform effort, will be led by Dr. Jason Kim (PI) of Systemic Research, Inc. 
An advisory committee with representation from other major national reform 
initiatives will provide further review and consultation in order to situate the 
proposed study in the national context. 



Dr. Jason J. Kim (PI) /Systemic 
Research, Inc. 

Dr. Rolf Blank (Co-PI) /Council of 
Chief State School Officers 

Dr. Pendred Noyce (Co-PI)/Noyce 
Foundation 

Dr. Lloyd Richardson (Co-PI) / 
University of Missouri- St. Louis 

Dr. Melva Ware /University of 
Missouri- St. Louis 

Dr. Hunhee Ken Lee /Systemic 
Research, Inc. 

Ms. Jennifer Manise /Council of Chief 
State School Officers 

Mrs. Linda Crasco /Systemic Research, 
Inc. 

Mr. Don Lee /Systemic Research, Inc. 



Dr. Lloyd Cooke /LMC Association 

Dr. David Grissmer / RAND Corp. 

Ms. Sharon Lewis /Council of Great 
City Schools 

Dr. William Schmidt /Michigan State 
University 

Ms. Clara Tolbert /Philadelphia USI 

Dr. Yuwadee Wongbundhit/Miami- 
Dade USI 



NSF Program Directors: 

Dr. Bernice Anderson /Division of 
Research, Evaluation and 
Communication, Directorate for 
Education and Human Resources 

Ms. Celestine Pea /Division of 
Educational System Reform, Directorate 
for Education and Human Resources 



Evaluative Study Team 



Advisory Panel Member 
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